הלכה על אבות 1:6

And all this even if the story were about a plain man, who usually was careful not to sin, but sometimes stumbled into it. How much more so if it were about a G–d-fearing man, to whom there applies even more the mitzvah of "In righteousness shall you judge your neighbor" (viz. Rambam on Avoth 1:6 and Rabbeinu Yonah, Sha'arei Teshuvah 218). And he [the hearer] transgresses this and judges him in the scales of guilt, agreeing with the speaker in his demeaning of him! Of a certainty he transgresses the issur of accepting lashon hara.